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This paper presents a lodel for evaluatin;g; stiident 
that emphasizes the interplay between the paper and the 
iatorrsp-4st studies of coiposition evaluation have tended to 
focus on the^pltpei^r on the rater, but not both. The paper concludes 



that further experxiental research, on the evaluation process lust be 
designed to examine the entire context of composition evaluation^ 

jmi^ ^ ^ ^ ""^ 
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One pressing concern in English education today a's_s.tudent 
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teachers in the United States, interpreted the lates;6i''^|ul'ts of 
the- Sationai Assessment of Educational Progress as ''dilljurijing " 
evidence ^hat at least, some of the best high school students are 
under stimulated %y the writing curriculum, and t:iat there is 
wholesale s-J;agnation -throughout the high school years among the 
least able writers" (Mellon,. \l975, p.2'9). They declared 1976 
■ th'e year of composition and vbwed to do "soiaething" about the 
crisis- in writing. Newsweek .Hagazine * s i)ec^mbier 8, 1975, 
cover story., "V/hy Johnny Can H^r it e," popularized the problem,"" 
stimulatirrg public ag. well as professional 'concern. 

The problems in the field of composition go far beyond 
whether or not students vrrite mori poorly now than they did in 
the past. Many tecchers question Vtheir own ability and, for 

t y 

that matter, anyone's ability to ^tfeach composition. , Furthermore, 
those cprivinced that they can and do teach students to improve 
their writing ha^e difficulty demonstrating their convictions. 
The subjectivity of the composition\evaluation , process frequently 



.results in erratic evaluations of student papers by" teachers 
ahci by researchers. -Joseph V/illi£.us in a Jceynote address at the 
-annual meeting' of of'^TE in 1976 emphasized how _little researchers 
knov/ about how and why e valuators or teachers judge student • 
papers as they do. This inability to evaluate compositions 
has k§pt both pedagogyland research .at a primitive level. , \ 

Mos^t of the pas.t. /attempts to solve problems in evaluation 
have either failed to answer or have ignored a central question:./ _ 
Why, do evaluators award the ratings they do to student papers? 

To answer tbls question researchers need to identify the factori 

^ / 

^in the papers, the raters, and the context of'^ the evaluation t^at 
\p.nfluence the evaluation. \ , ' 

/. / Hiller,- Marcotte, and Martin (l969) after \completing\a 

study of student essays suggested: \ • . 

\ if a given characteristic is present in an essay, 

\ does that characteristic affect the e's^sty^s \ 

\ quality as reflected in the grade a^Lsigried by 

\ fexpert graders. To answer .this^ (Ju^e'sti^n we should 

\ have to manipulate the quality -and' qua^niity of \ 

\ relevant ^category items under a^-^experimentali 

^Procedure, (p. 274) ^ | ' \ 

\ ^ ^ ' ' ^ ^ ' ' 

Most past r esearch attempting to i identify the factors 

influencing raters^ judgments has been correlational and, 
therefore, , could hot establish patterns of influence^. Experi- 
mental Wesearch is needed. i rxj^ 

The correlational s.tudies 1-n composition evaluation, 
furthermore, l^ave tended to":fDT3us on the paper or the;r^ter, 
but not both. Paper-focused - studies e.g. Page (196'8)> 
Slotnick and Knapp^.(l97l), Hiller. et all,. ' (1969), Thompson {1^1 6) , 



and.Nold and Ereedraan (PortKcoMng) coiirj^lated^ propei:ties of 
the student papers such as .the number of spelling errors and 'tire- 
length Gf essay v/ith ratings of those papers, ' Rater-focused 
studies [e.g. Diaderich et al. (19&1) and Meyers et.al. (1966)] 
correlated characteristics of the raters such as personal bias(^s 
and degree of leniency with' their ratings.. ^ / 

; Part A of ?igure^ l depicts the assumptions of the paper- '[ 

focused, studies. The rater is, outside the bounds of the 
evaluation process because all raters ^re ^expected to .give the 

^jgame response, much like a computer. Part B of Figure 1 depicts 
tlie assumptions of the rater-focused studies. The paper remains 



•outside the p^rocess. Disagreement between ^raters results from 
;Lt'heir' biases and other personal qualities. , . . » 

.V . ■ ' 

Figure 2 provides .a more comprehensive model on which 
expei.imental studies could be based.. Both the evaluatbr and the 
paper are central to the process* The i\ecipro,cal arrow connecting 
the rater and paper represents an interplay b'^tween the two. The 
qualities in the paper are* meaningful in terms of how. the evalu- , 
_^or perceives them. The evaluator creates part, of the meaning, . 
of the paper, but qualities intrinsic ^;o thfe paper set bounds^ 
pn the. meaning the evaluator may create*. Most of the time the 
rating is based on this interplay. However, the line cohnecting 
th^^ rating with the paper is dotted to allow for ratings that 
are n6t based totally oh^ (iualities within the paper* Rosner, 
for example, revealed that raters gave higher marks to papers 
they thought writ.ten ?by honors ^students than those they thought 
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written by average students (in Diederich, 1974, pp#\ 11-15) • 

How^ the ..evaluator perceives and reacts to the papef is 
modulated by personal characteristics resulting f rom intrinsic 
qualities 'Sihd past;^nd present experiexices as wall as by a 
cifcumstantiail' environment manifest at the time of the reading. ■ 
These personal chara'cteris tics likely to be important to, the 
evaluation inclji;de4 aptitudes, education, teaching and writing 
. experience,t,^ priiorvkiipwledge about* the writer , and .personal anxi- 



eties and tensions • The circumstantial environment , the immedi- 

\ ■ \ 

ate context of the reading, consists of features like: training; 
time of day; lengt^ of task;; physical environment — comfort of 
chairs ,, lighting of xoom, heating, jioise level; the nature of , 
the task — the kind of rating scale, types or^^pers to be read 
(narrative, expository), context of other papers, ^inferences 
about the writers based on reading,, the, paper*. The shaded 
area in Figure 2 illustrates the melding of these personal 
cHaracteristics with the cir cums tan tial environment during 
the'^valuation ^roce ^ [ / ^ " 

'Experimental research on the eva|tuation pracess must*ba 

i ' . ■ ' V J ' - 

designed to allow ejxamination of the entire context of the 

"v\ ^ ^ ^ - : 

evaluati^n--a study of the interplay^between the^ paper and 
the evalu^or* .1 propose that the eyaluator^a qualities in- 
conjunction\with the paper itself determine his or fier per-, 
ceptioh and dyaluatioh lof the paper* The evaluator^s perception 

of the paper, in a particular c6n.text, wisLtten by a particular 

" ~ " - - ^ / \ ^ . ' ..... , 

" ^ - - . . . 

writer, uiider paEyticular conditions, will determine the rating. 
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